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Joint experimentation that investigates sensor optimization, re-tasking and management has far reaching implications for 
Department of Defense, interagency and multinational partners. An adaption of traditional human in the loop (HITL) Modeling and 
Simulation (M&$) was one approach used to generate the findings necessary to derive and support these implications Here an 
entity-based simulation was re- engineered to run on USJFCOM s High Performance Computer (HPC). The HPC was used to 
support the vast number of constructive runs necessary to produce statistically significant data in a timely manner. Then from the 
resulting sensitivity analyses, event designers blended the necessary visualization and decision making components into a synthetic 
environment for the HITL simulations trials. These trials focused on areas where human decision making had the greatest impact 
on the sensor investigations. Thus, this paper discusses how re-engineering existing M&S for constructive applications can 
positively influence the design of an associated HITL experiment. 


1.0 INTRODUCTION 

United States Joint Forces Command 
(USJFCOM) Joint Concept Development 
and Experimentation Directorate (JCD&E) 
develops innovative joint concepts and 
capabilities providing experimentally proven 
solutions to the most pressing problems 
facing the joint force. JCD&E mitigates risk 
for DoD through rigorous evaluation of 
alternatives and through the development, 
testing and validation of joint concepts 
focused on specific problems identified in 
the Joint Operating Environment or gaps in 
doctrine. Joint experimentation is 
complementary with other elements of DoD 
research, development, testing and 
evaluation offices and applies similar 
methods to those used in technology test & 
evaluation and field demonstration. J9 
leads and coordinates JCD&E for DoD 
through an enterprise approach, applying 
structured, disciplined and transparent 
processes that maximize effectiveness and 
efficiency {JCD&E, 2010). 

1.1 Useful Definitions & Concepts 

A model is a description of the underlying 
methodology used to conduct an 
investigation, and a simulation is the 
implementation of that model that can occur 


via automation {Akst, 2010). Further, Akst 
wrote that simulations should not be built 
before developing the underlying models, 
and that the first step in successful model 
development must be to understand the 
intended use of both the models and the 
simulations (M&S). 

1.2 Intended Use of the M&S 

One JCD&E sponsored experiment was the 
Joint Integrated Persistent Surveillance 
(JIPS) project. This experiment included 
constructive simulation (CS) and Human in 
the Loop (HITL) activities which were 
instantiated on USJFCOM’s High 
Performance Computer (HPC). Based on 
the analysts’ developed metrics, M&S 
generated the environments necessary to 
allow analysts to capture data associated 
with those metrics. First, metrics were 
generated with the G2 process model 
(Snyder, 2010). These metrics represented 
the human decision making process, where 
model outputs were used to initialize the 
simulation, so that human interactions were 
not required during execution of the CS 
runs. Then for the HITL trials, solutions 
were investigated with the introduction of 
humans, as an alternative representation 
with complementing strengths and 
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weaknesses to the CS approach, which 
assisted in validating the modeled 
representations of the decision-making 
process. The CS provided a means to 
conduct a high number of trials not only for 
increased precision in the statistical output, 
but also to manipulate an increased number 
of solution set variables. Within the HITL, 
the number of trials was significantly 
decreased; however, the advantage was in 
the interaction of real people with the 
solutions. Insights that the analysts gained 
from the CS runs were used to assist event 
designers in determining the necessary 
M&S configurations for the associated JIPS 
HITL trials. Results from the HITL were 
used to assist designers in determining the 
setup of additional CS runs in keeping with 
a model-wargame-model (W-M-W) 
paradigm (Kass, 2006). 

1.3 Event Design 

The JIPS HITL provided experimental 
evidence to assist in determining the 
effectiveness and efficiencies of solutions 
against current day and futuristic baselines. 
Prior to the HITL runs, thousands of faster 
than real time CS runs were executed to 
examine the impact of the various solutions. 


Then during the HITL trials, solutions were 
carried forward with the introduction of a 
human decision-making component to 
further examine the solutions with the 
introduction of command, control, 
communication, computer, and intelligence 
(C41) systems. 

1.3.1 Experimentation Audience 

Figure 1 illustrates the four echelons of 
command that were emulated during the 
HITL. These echelons were: national to 
include combatant command (COCOM), 
joint task force (JTF), division (DIV) and 
regimental combat team (RCT) with 
associated tactical elements such as 
battalion and companies. In turn, parts of 
seven levels of Command and Control (C2) 
were emulated by these four echelons. The 
blue shaded area represented the 
experiment audience, and the pink shaded 
area represented the control cell with role 
players. The control personnel, called the 
White Cell (WC), ensured that the 
simulation rendering of the enemy activities, 
friendly assets and neutral ground activities 
were sufficient to stimulate the participants’ 
behavior. The analysts observed these 
behaviors and gathered data from the 



Figure 1. Separation of participants and simulation 
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simulation to develop their findings in 
keeping with the Data Collection and 
Analysis Plan (DCAP). M&S operators 
resided in the WC to assist in masking the 
M&S activities from the participants in order 
to enhance the realism of the experiment. 

1.3.2 Communications 

In addition to providing data in support of 
analysis, the simulation was also the data 
source for the integrated C4I systems which 
were used by experiment participants to 
carry out their duties. The M&S was used to 
facilitate sensor asset re-taskings, full 
motion video (FMV) taskings and system 
status reporting. Internet Relay Chat (mRIC) 
was the primary mode of communication 
among M&S operators, analysts, collection 
managers, targeteers and sensor asset 
managers at all echelons. Command and 
Control Personal Computer (C2PC) and 
Command Post of the Future (CPOF) 
presented participants with a shared 
representation of battle space entities as 
generated by the simulation. Email was the 
secondary communication channel that was 
available at all four echelons to send and 
receive lengthy textual reports such as 
formatted intelligence reports. Additionally, 
voice over IP (VOIP) provided a digital voice 
communication capability at all JIPS HITL 
workstations. These HITL workstations 
emulated those of real world C2 centers. 

2.0 BODY 

Supporting the HITL were federation M&S 
components that stimulated the C4I tools. 
These stimuli evoked behaviors from the 
participants which were of interest to the 
analysts. 

2.1 Federation Components 

There were six primary M&S components 
used in the HITL to generate situational 
awareness (SA) for the participants, and 
five of these components directly drove the 
Common Operating Picture (COP). First, 
Joint Semi-Automated Forces (JSAF) was 
the entity level simulation that represented 
the opposing entities. These entities were 
controlled by human operators and the 


JSAF automated behaviors. Second, 
CultureSim, a scalable entity level federate 
tightly coupled with JSAF that was formerly 
known as ClutterSim, represented the 
neutral populace reflected within the urban 
battlespace (Speicher, 2004). Third, 
Simulation of the Locations and Attack of 
Mobile Enemy Missiles (SLAMEM) was the 
entity level simulation used to represent 
sensors, sensor platforms, fusion, and 
tracking. Fourth, a redesigned Track 
Database (TrackDB) federate enabled direct 
communications from JSAF to the C4I 
components. This modified TrackDB 
federate replaced the need for a separate 
C41 gateway to feed the Global Command 
Control System - Joint (GCCS-J) server. 
C2PC and CPOF clients pulled data from 
GCCS-J. Fifth was the three dimensional 
JSAF viewer (JStealth), which produced 
effects that closely resembled those from 
emulated platforms generating FMV feeds. 
Finally, an experimentation & event tool suit 
(EETS) component, the Event Generator 
(EventGen), facilitated the injection of 
master scenario event list (MSEL) items. 

The EventGen injects supplemented the 
perceived behavior of the simulation 
entities. These injects were represented as 
emails that did not directly influence the 
COP, but did provide amplifying context and 
intent to further influence the players' 
reaction to the M&S stimulation of the COP 

2.2 C4I and M&S Architectures 

M&S operational requirements for the JIPS 
HITL were broken out into the five areas. In 
turn, four technical spirals and one 
operational test were used to ensure that 
these five areas were sufficient to 
successfully accomplish the experiment 
from a technical standpoint (JIPS, 2010). 
These tests were conducted to ensure that 
surveillance and network architectures, 
simulation and communication components, 
plus the control procedures were sufficient 
to meet event objectives. Figure 2 depicts 
how the M&S components interfaced with 
the COP and the FMV view generators to 
stimulate the participants. Not all message 
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traffic was processed through the TrackDB, 
Over-the-Horizon (OTH) Targeting Gold 


to obscure the enemy activities that were 
represented in JSAF, MSEL items 


Read in the MSELs 



Figure 2. Simulation and C4I Architectures 


{OTG) messages passed initial detections 
directly from SLAM EM to the GCCS-J 
server. The perceived current track states 
were sent via the TrackDB. Objects and 
interactions from SLAMEM and the 
TrackDB were pulled from the Federation 
and placed into a MySQL repository for later 
analytical use. In some cases, SLAMEM 
data were captured in SLAMEM logs, and 
other Federation traffic was pulled via a 
separate application called JLogger. 
TrackDB was enhanced for this event to 
send track data directly to the GCCS 
system, removing the need for a separate 
C4I gateway. The JSAF Surveillance Tool 
was the mechanism used to allow re-tasking 
of sensor assets. 

2.2.1 Re-tasking Sensors Assets 

During the HITL, participants tasked sensor 
assets to collect information on perceived 
enemy activities as instantiated in JSAF. 
Closely coupled with JSAF was the 
CultureSim federate which generated 
thousands of neutral entities which served 


established the initial taskings for the sensor 
assets that were represented in SLAMEM. 
Once re-tasking requests were approved 
and authorized, HITL participants required 
the capability to influence airborne sensor 
assets to change the location of interest 
view, the assigned sensor on location, 
and/or fly a new route to meet a new or 
modified collection requirement. From an 
M&S perspective, blue sensor assets, 
represented in SLAMEM, had to be capable 
of changing their station, their surveillance 
area, or their sensor modes as directed by 
HITL participants. 

2.2.2 Observe FMV 

From an M&S perspective, FMV was 
simulated by attaching JStealth to simulated 
sensor assets that were represented in 
SLAMEM, The simulated FMV feeds, which 
were visualized through JStealth, were fed 
to a video server capable of streaming the 
video over the network to participants' C4I 
displays. HITL participants requested FMV 
feeds via the WC. Working within the WC, 
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the M&S operators changed the FMV views 
per the participants’ requests. 

2.2.3 Inject Scenario/MSEL Events 

The experiment executed a scenario with 
multiple storylines to provide target activity 
for persistent and non-persistent 
surveillance. Detailed events from each 
storyline were compiled into a single 
chronologically ordered event list known as 
the MSEL, which was executed at the 
discretion of the white cell. The WC 
controlled MSEL inject flow via EventGen. 
The control group also had the responsibility 
to add to or delete from the MSEL injects to 
ensure the experiment objectives were 
accomplished. 

2.2.4 Situational Awareness 

Situational awareness (SA), as applied to 
the JIPS HITL, was the capability to extract 
meaningful activities and patterns from the 
battlespace picture and to share this 
awareness across the network with 
appropriate participants {Hayes, 2006). 

The COP provided the cognitive 
understanding or interpretation of the JIPS 
battlespace. The JIPS battlespace was 
composed of the following four SA 
components (JIPS, 2010): 

• Blue Force SA component - a 
complete picture of all Blue force 
entities including their identification 
and position. Blue force sensor 
entities were modeled in SLAMEM. 

• Red Force SA component - a 
complete picture of all Red (hostile) 
force entities including their 
identification and position. Red 
force entities were modeled in JSAF. 

• Green Force SA component - a 
complete picture of all civilian 
units/entities including their position 
and (optional) identification 
information. The Green component 
was modeled in JSAF, which 
included cultural activities, i.e., 
people, vehicles, and landmarks. 

• Sensor SA component - a picture of 
the perceived entities sensed in the 


battlespace and reported by Blue 
force sensor systems. Blue force 
sensor reports were produced by 
SLAMEM. 

2.2.5 M&S Data Collection 

The JIPS HITL exercised the innate 
distributed collection mechanisms of the 
tools and logged all federation simulation 
traffic. The After Action Review (AAR) was 
supported by various means of 
consolidating the data from the HPC, data 
fusion techniques, and data reduction into 
user friendly formats as specified by the 
JIPS analysts. 

3.0 DISCUSSION 

3.1 Synthetic HITL Environment 

The JIPS HITL federation was composed of 
simulations selected to best meet the multi- 
echelon command structure of joint 
persistent surveillance operations. The goal 
of the JIPS technical and operational test 
program was to mitigate the risk inherent in 
the use of disparate simulation and SA 
components by investigating and evaluating 
the functionality and interfaces of the 
components within the framework of the 
event objectives. For the JIPS experiment, 
persistent surveillance was defined as a 
collection strategy that emphasizes the 
ability of collection systems to linger on 
demand in a particular location to detect, 
locate, characterize, identify, track, target 
and assess in real or near real time (HQDA, 
2010 ). 

3.2 M&S Architecture 

Various optimization techniques where 
employed to take advantage of the HPC 
architecture and address experimentation 
objectives. 

3.2.1 Entity Count 

In order to reach the JIPS HITL 100,000 
entity count target, several federation 
routers and CultureSim nodes where 
configured to take advantage of the M&S 
Data Distribution Management (DDM) 
protocol. This approach reduced the 
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number of interactions within the federation 
to a degree sufficient to support pockets of 
densely populated urban terrain areas. 

3.2.2 Gateway Redesign 

Given the probability that sensor collection 
plans would include fly-over of these dense 
populations, it was necessary to optimize 
the C4I Gateway to support the simulation- 
to-C4l translation of large numbers of 
perceived tracks. This was resolved by, 1) 
redesigning the TrackDB federate to replace 
the C4I Gateway, effectively removing the 
less efficient runtime database access 
requirement and, 2) bundling and 
distributing consolidated sensor reports as 
opposed to individual tracks, 

3.2.3 Data Collection 

During runtime, a distributed data collection 
approach was used. Collection was 
localized at each simulation/HPC node 
reducing the runtime data throughput 
requirement. After each HITL run, the data 
was off-loaded from the HPC onto a 


the nodal scheme: yellow blocks 
represented JStealth nodes, green blocks 
represented CultureSim nodes, purple 
blocks represented SLAM EM nodes, grey 
represented router nodes, red blocks 
represented JSAF nodes, and the white 
blocks represented the simulation that held 
the track histories. The node reference 
numbers within the blocks corresponded to 
256 nodes that were mapped to a particular 
application. Blocks without node references 
represented stand alone components. Due 
to the heavy data traffic on the HPC, several 
nodes were dedicated to routing the traffic. 
Additionally, a majority of the HITL nodes 
were dedicated to instantiating JStealth and 
CultureSim due to their high computational 
demands. 

3.3 Operational M&S Requirements 

The approach to operational testing began 
with identifying a set of operational 
requirements having functional 
dependencies on the M&S. In turn, the CS 
runs were used to assist the setup of the 



Figure 3. HPC Network Connectivity Map 

centralized repository for analysis. Colors HITL simulation trials. The primary 

used in figure 3 highlight the complexity of difference between the CS and HITL runs 
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was the human interactions by the 
participants. Observations made of these 
actions were used to validate the human 
decision making models. Other differences 
were associated with the addition of M&S 
and C4I to stimulate components that 
provided credible visualization and decision 
support aids to assist the participants in 
making decisions. 

3.3.1 Constructive Inputs 

From the CS, insights into how the FMV 
views should be synchronized with the 
MSEL injects resulted in multiple screen 
views for the participants. The information 
gathering and fusion processes, associated 
with the FMV views, were implicitly 
performed during the CS runs. By using the 
same sensor simulation for both the HITL 
and CS runs, this commonality assisted 
event designers in establishing traceability 
between the CS and HITL results. 

3.3.2 Scenarios 

Traceability was reinforced by using the 
MSEL descriptions of the scenario’s 
storylines to develop the synthetic 
representations of the people, enemy forces 
and the necessary landmarks to stimulate 
the participants such that they would re-task 
assets as SA dictated. In other words, the 
JIPS event designers used the implicitly 
represented actions in the CS runs to 
determine the appropriate actions that the 
human participants would explicitly need to 
execute during the HITL runs. Simulation 
time was maintained in a MySQL database 
(Figure 2). The EventGen tool kept the 
MSELs synchronized with the simulations 
by reading and writing to this database, 

3.3.3 Refining Models 

Per the M-W-M paradigm, the first set of G2 
models were based on expert judgment to 
develop time estimates that predicted the 
participants’ performance during execution 
of each solution in the HITL. Next, HITL 
observations were used to update the 
experts’ time estimates, which led to refining 
the G2 models for the second series of 
model runs. 


4.0 CONCLUSIONS 

The JIPS HITL was conducted to provide 
experimental evidence on the effectiveness 
and efficiencies of potential solutions to the 
persistent surveillance problem. The 
implementation of the HITL experiment on 
the USJFCOM HPC was necessary to fully 
implement the M-W-M paradigm. With the 
ability to rapidly refine the G2 models based 
on metrics derived from the HITL, the HPC 
made the re-running of the CS feasible 
within the project's time constraints. Thus, 
event designers used the HPC to achieve a 
closer coupling between the modeling and 
wargaming trials than what was previously 
possible in other JCD&E M&S supported 
activities. This coupling came from using 
the output data of the HITL to further refine 
the G2 models used for a second set of CS 
runs. 

4.1 Constructive Engineering 

Based on the results of using a version of 
SLAMEM in a constructive simulation mode, 
lessons were learned which influenced how 
best to custom build a JCD&E experiment 
with M&S and C4I components. This 
environment was sufficient to provide 
participants with the SA necessary to make 
meaningful decisions in line with the desires 
of the analysts. Observation tools and 
surveys were used daily, and the simulation 
generated metrics provided context which 
allowed traceability back to the DCAP. The 
use of common MSELs, between the CS 
and HITL trials, became an important factor 
for facilitating traceability as well as saving 
the time necessary to create a different set 
Of MSELs. 

4.2 Resources 

Wthout the M&S environment created by 
the USJFCOM HPC, dozens of additional 
stand alone servers and operators would 
have been needed to correlate the answers 
and findings. Additionally, from the resulting 
sensitivity analyses event designers were 
able to select the sufficient type of 
visualization and decision making 
components to create the necessary HITL 
environment. It was the application of the 
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USJFCOM HPC resource that allowed for 
the focused and timely manipulation of 
simulation data, routing of data traffic, and 
stimulation of the C4I systems that provided 
the human decision makers a credible 
environment for JIPS investigations. This 
manipulation of data was sufficient to 
provoke the participants to take actions that 
were of interest to the analysts. 

4.3 Optimization Techniques 

Intelligently exploiting the M&S DDM 
protocol on the HPC led to the reduction of 
inter-node simulation interactions thus 
reducing network traffic and unnecessary 
message processing. Additionally, 
localizing the collection of data at each 
simulation/HPC node reduced the runtime 
data throughput requirement. After each 
HITL run, the data was off-loaded from the 
HPC onto a centralized repository for 
analysis. However, it was the redesign of 
the Track DB federate and bundling of 
sensor reports that proved to provide the 
most benefit to the participants. This was 
because the redesign and reconfiguration 
reduced C4I track latencies, and contributed 
most to an environment that dispelled 
disbelief in a constructively engineered 
world. 
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